Local Boundary Regularity of the Szegő Projection and Biholomorphic Mappings of Non-pseudoconvex Domains

It is shown that the Szegő projection S of a smoothly bounded domain Ω, not necessarily pseudoconvex, satisfies local regularity estimates at certain boundary points, provided that condition R holds for Ω. It is also shown that any biholomorphic mapping f : Ω → D between smoothly bounded domains extends smoothly near such points, provided that a weak regularity assumption holds for D. 1. Preliminaries Throughout, Ω denotes a smoothly bounded domain in C and r a C∞ defining function of Ω. The notation W (Ω), s ∈ R, stands for the Sobolev space of order s. The closure of C∞ 0 (Ω) in W (Ω), s > 0, is denoted by W s 0 (Ω) with dual space W−s(Ω). The norm of W−s(Ω) is then ‖u‖−s = sup{|〈u, φ〉|;φ ∈ C∞ 0 (Ω), ‖φ‖s = 1}, u ∈W−s(Ω). Also the dual space (W s(Ω))∗ of W (Ω) is defined, and the norm of u ∈ (W s(Ω))∗, is ‖u‖−s = sup{|〈u, φ〉|;φ ∈ C∞(Ω̄), ‖φ‖s = 1}. Certainly (W s(Ω))∗ ⊂ W−s(Ω), and ‖u‖−s ≤ ‖u‖−s for u in (W s(Ω))∗. However, if u is harmonic then ‖u‖−s ≤ C‖u‖−s with C independent of u. Following Boas [12], the norm ‖ · ‖(∗) s is defined to be ‖ · ‖s if s ≥ 0, and ‖ · ‖∗s if s < 0. W (∂Ω), s ∈ R, denotes the boundary Sobolev space. Any function in W (∂Ω) is identified with a harmonic function in W (Ω) via the Poisson integral with equivalent norms: C−1‖u‖s+1/2 ≤ ‖u‖Ws(∂Ω) ≤ C‖u‖s+1/2, (1.1) where C is a constant independent of u (The letter C in this paper denotes a positive constant which may vary at each of its occurrences.) There is also a local equivalence of these two norms. If ζ1, ζ2 are in C ∞ 0 (C) and ζ2 ≡ 1 near supp ζ1, the support of ζ1, then ‖ζ1u‖s ≤ C(‖ζ2u‖ Ws−1/2(∂Ω) + ‖u‖ W−M (∂Ω)), ‖ζ1u‖Ws(∂Ω) ≤ C(‖ζ2u‖s+1/2 + ‖u‖−M), (1.2) Received by the editors September 25, 1995 and, in revised form, July 30, 1996. 1991 Mathematics Subject Classification. Primary 32H10. c ©1998 American Mathematical Society